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About This Guide 


This guide describes how to use Novell Cluster Services in VMware virtual environments. The 
implementation guidelines include recommended settings and administrative processes to deploy 
and use Novell Cluster Services in virtual environments. 


* Chapter 1, "Getting Started with Novell Cluster Services in an ESXi Virtualized Environment," on 
page 7 


* Chapter 2, "Planning for the Virtualized Environment," on page 17 


* Chapter 3, "Implementing Novell Cluster Services in an ESXi Virtualized Environment," on 
page 21 


* Appendix A, "Documentation Updates," on page 57 
Audience 


This guide is intended for system administrators who are familiar with VMware technology and Novell 
Cluster Services. A basic understanding of VMware software and virtual environments is assumed. 


Feedback 
We want to hear your comments and suggestions about this manual and the other documentation 


included with this product. Please use the User Comments feature at the bottom of each page of the 
online documentation. 


Documentation Updates 


For the most recent version of the Novell Cluster Services Implementation Guide for VMware, visit 
the OES 11 SP3 documentation website (http://www.novell.com/documentation/oes11/). 


Additional Documentation 


For information about managing Novell Cluster Services clusters and resources, see the OES 11 
SP3: Novell Cluster Services for Linux Administration Guide. 
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Getting Started with Novell Cluster 
Services in an ESXi Virtualized 
Environment 


Novell Cluster Services (NCS) for Novell Open Enterprise Server (OES) 11 Support Pack (SP) 3 
supports clustering virtual machines, physical machines, and a mixture of virtual and physical 
machines in the same cluster. Among other benefits, clustering virtual machines can help reduce the 
hardware costs of traditional clusters. 


The ESXi 5.1 hypervisor, the latest virtualization software developed by VMware, allows multiple 
virtual machines on a single host. ESXi supports the latest processors from Intel and Advanced Micro 
Devices. The hypervisor abstracts the underlying bare-metal hardware, which allows you to define 
virtual machines with the virtual hardware needed by a guest operating system. 


This guide describes how to get the most from Novell Cluster Services within an ESXi virtualized 
environment. 

¢ Section 1.1, "Configuration Overview,” on page 8 

¢ Section 1.2, "Understanding Virtualization,” on page 8 

¢ Section 1.3, "Architectural Scenarios,” on page 11 

¢ Section 1.4, "Design and Architecture Considerations," on page 14 
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1.1 


1.2 


Configuration Overview 


A typical Novell Cluster Services cluster (see Figure 1-1) consists of a number of machines that 
coordinate through computer networks to make resources highly available. Nodes in a cluster can be 
physical, virtual, or any combination of the two. The cluster resources often reside on shared storage. 


Figure 1-1 Typical Cluster Configuration with Virtual and Physical Machines 
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A typical ESXi host machine includes virtual network switches, disk mappings, and virtual machines. 
A virtual machine can access the network directly through PCI pass-through, or through virtual 
switches inside the ESXi host. 


Just like other Linux systems, shared storage is presented in ESXi hosts as devices. Disk mapping 
provides a way for virtual machines to share those devices. Clustering of virtual machines normally 
requires the mappings of shared devices to be Raw Disk Mappings (RDM). The RDMs cannot reside 
on a datastore on shared devices, unless you have lower-level cluster software to manage the 
datastore. 


From the NCS point of view, there is no difference between a virtual machine and a physical machine. 
You can cluster any combination of them. Although NCS supports clustering mixtures of physical 
machines and virtual machines, it is easier to manage clusters where all nodes have the same or 
similar capacity. 


Understanding Virtualization 


Virtualization can be used at multiple computing levels to provide services for your enterprise. 


¢ Section 1.2.1, “Where Is Virtualization Today?,” on page 9 
¢ Section 1.2.2, "Why Use Virtualization?,” on page 9 
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1.2.1 


1.2.2 


1.2.3 


1.2.4 


* Section 1.2.3, "Why Use Novell Cluster Services?,” on page 9 
* Section 1.2.4, "Server versus Service Virtualization,” on page 9 


Where Is Virtualization Today? 


Virtualization of servers and services is everywhere: 


* Virtualization with hardware, such as blade centers or Cisco Unified Computing System (UCS). 
¢ Virtualization with software, such as VMware, XEN, KVM, Hyper-V, and more. 
* Virtualization within the infrastructure: 

* Network: VLAN 

* SAN: port virtualization in the SAN 


* Storage virtualization 


Why Use Virtualization? 


The benefits of virtualization include the following: 


* Allows consolidation of servers to help reduce costs for hardware and power. 


* Allows independence of servers from hardware and environmental complexities and 
infrastructure. 


* Allows scalability of services on a single server because the hardware is more powerful than one 
service needs. 


* Increases server and service availability. 
* Allows you to use cloud services. 


* Builds independence from hardware to gain the flexibility of managing hardware connectivity on 
only the virtualization level. 


Why Use Novell Cluster Services? 


The benefits of using Novell Cluster Services as your clustering solution include the following: 


* Increases service availability, minimizing recovery time when problems occur 
* Consolidation 

* Flexibility 

* Scalability 

* Manageability 

* Hardware independence 


Server versus Service Virtualization 


Virtualization can occur on different levels in the computing environment. The difference between a 
server and a service is summarized in Table 1-1. A server consists of hardware, an operating system 
environment made up of the kernel and hardware drivers, one or more services, and a process 
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scheduler to control the services. A service consists of data and the service configuration settings, the 
application that provides the service (including its process, code, and executable), and a network 
address to give users access to the service. 


Table 1-1 Server versus Service 


Server with Operating System Service 
* Hardware * Data and configuration 
* Operating system (kernel, hardware drivers) * Application (process, code, executable) 
+ One or more services * Network address 


* Process scheduler 


In a virtualized environment, a server can be virtualized as a virtual machine that is independent of 
the hardware. Virtualization hypervisors such as Xen, KVM, VMware, and Hyper-V allow virtualization 
of servers. 


Figure 1-2 Virtualization Stack 
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With Novell Cluster Services, a service can be virtualized as a cluster resource that can be failed over 
between nodes. Clustering makes the service independent of the hardware. 


Figure 1-3 Server Stack 
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1.3 


1.3.1 


Architectural Scenarios 


Typical architectural scenarios for virtualized environments are illustrated in this section. You can use 
server and service virtualization at different levels to achieve your networking goals. 


* 


* 


* 


* 


* 


Section 1.3.1, "Only Service Virtualization," on page 11 

Section 1.3.2, "Only Server Virtualization," on page 12 

Section 1.3.3, "NCS on Host Machines Managing Xen or KVM Guest Machines," on page 12 
Section 1.3.4, "NCS Managing Services on Guest Machines," on page 13 


Section 1.3.5, "NCS Managing Services on a Cluster of Physical and Guest Machines," on 
page 13 


Only Service Virtualization 


Novell Cluster Services can be installed on physical machines. Services are virtualized as cluster 
resources that can fail over between physical machines in the same cluster. 


Figure 1-4 Service Resources on Physical Servers 
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1.3.3 Only Server Virtualization 


Guest machines can be hosted on physical machines. No services are virtualized. 


Figure 1-5 Guest Servers with No Clustering 
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1.3.83 NCS on Host Machines Managing Xen or KVM Guest 
Machines 


Novell Cluster Services can be installed on host machines. Virtual machines are virtualized as cluster 
resources that can be failed over between physical machines in the same cluster. Nodes in the 
cluster run the same virtualization software, such as Xen or KVM. 


Figure 1-6 Virtual Machines as Cluster Resources on Xen Hosts 
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This scenario is covered in “Virtual Machines as Cluster Resources” in the OES 11 SP3: Novell 
Cluster Services for Linux Administration Guide. 
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1.3.4 


1.3.5 


NCS Managing Services on Guest Machines 


Novell Cluster Services can be installed on guest machines. Services are virtualized as cluster 
resources that can fail over between virtual machines in the same cluster. 


Figure 1-7 Guest Machines as Cluster Nodes 
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NCS Managing Services on a Cluster of Physical and Guest 
Machines 


Novell Cluster Services can be installed on guest machines or physical machines. Any combination of 
physical machines and virtual machines that are running the same operating system can be in the 
same cluster. Services are virtualized as cluster resources that can fail over between the nodes in the 
same cluster. 


Figure 1-8 Physical and Guest Machines as Nodes in a Cluster 
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14 Design and Architecture Considerations 


Before you deploy virtualized solutions, you should understand common architectures as well as the 
benefits, caveats, pros, and cons of virtualization for servers and services. 

¢ Section 1.4.1, "Challenges of Server Virtualization,” on page 14 

¢ Section 1.4.2, "Cost of Server Virtualization," on page 14 

¢ Section 1.4.3, "Challenges of Service Virtualization," on page 15 

¢ Section 1.4.4, "Cost of Service Virtualization," on page 15 

¢ Section 1.4.5, "Fault Tolerance and Scalability,” on page 15 

¢ Section 1.4.6, "Planning Considerations," on page 15 

¢ Section 1.4.7, "Full Virtualization versus Paravirtualization," on page 15 


¢ Section 1.4.8, "Comparing Architectures,” on page 16 


14.31 Challenges of Server Virtualization 


* Server virtualization versus service virtualization 
* Operating system virtualization overhead 
* Disk space used, memory, and CPU 
* Maintenance (install, configure, upgrade) 
* Service virtualization has more dependencies 
* Multiple guests on the same host -- how to do quality of service 
* Hardware monitoring--on which level 
* Hardware monitoring with NCS means a service restart (faster) 


* Hardware monitoring with a virtual machine host means a server restart, with a potentially 
higher risk of corruption for the virtual machine. 


* Service monitoring--on which level 
* Consolidation--on which level 


* Resource migration--on which level 


142 Cost of Server Virtualization 


* Server virtualization has more overhead than service virtualization because of emulation of 
server hardware (main board, NIC, and HBA). 


* Server virtualization has up to 8096 performance loss when compared to native installation on 
hardware. 


* When do we realize the performance loss? 


* Only during high load I/O and not during normal work, such as GroupWise maintenance or 
database work. 


* During backup, restore, and data migration. 
* Whenever we have a high number of small I/O. 


* Because of the virtualization, higher latency and more CPU usage. 
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14.3 Challenges of Service Virtualization 


* Same patch level for all services is required on one host. 

* Dependencies exist between services on the same host. 

* Itis impossible to run the same service twice on the same server (isolation). 
* Processes are not designed to run twice on the same server. 


* Running multiple instances can cause networking conflicts (ports). 


144  Costof Service Virtualization 


* Fewer physical servers and less hardware are required. 
* Less maintenance for the physical server's OS is required. 


* One service can cause a server or other services to crash. 


1.45 Fault Tolerance and Scalability 


* NCS is made to monitor hardware and to monitor services. 


* VMware is made to monitor hardware and servers. Doing the same thing on multiple layers 
results in sub-optimal architecture. 


* Creating multiple cluster nodes on the same virtualization host does not provide the same 
hardware fault tolerance as having the nodes on separate servers. However, if a guest server 
fails, you can fail over the resource to another guest server. 


* Creating multiple clusters within a virtualization environment allows you to maximize the use of 
your hardware, but it does not provide the same hardware fault tolerance as having clusters 
running on separate servers. 


1.46 Planning Considerations 


* Performance 
* CPU and memory 
* Networks and storage 
* Fault tolerance 
* No single point of failure outside the hosts 


* Resource fail-over matrixes can help assure that you have provided a fail-over environment 
for each resource 


* Balancing workload 
* eDirectory servers 


For information about configuring and managing eDirectory servers in your OES 11 SP3 
environment, see the NetIQ eDirectory 8.8 SP8 documentation website (http:// 
www.netiq.com/documentation/edir88). 


* The cluster master 


1.47 Full Virtualization versus Paravirtualization 


* Paravirtualization drivers for NIC and SCSI 
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* Only SLES 11/ OES 11 

* Manageability 

* vMotion or virtual machine migration between hosts 
¢ Virtual disks versus raw disk mappings 

* PCI pass-through 


1.4.8 Comparing Architectures 


* 20 services need to run on 4 physical servers 
* Advantages and disadvantages of these solutions 
* Installation, configuration, and change management 
* Just NCS hosting 20 resources 
* Does not have the overhead of a hardware virtualization layer 
* Has hardware and service dependencies 
* Just a hypervisor hosting 20 virtual servers 
* Maintenance overhead 
* Virtualization overhead 
* Hypervisor hosting 4 servers with NCS that has 20 resources 


* Advantages of both the NCS solution and hypervisor solution 
The benefits of using both virtualization and NCS include: 


* Combines server and service virtualization in an intelligent way. 
* Balances the number of servers with the number of services. 
* Uses the same rules as NCS on physical environments. 


* NCS allows you to run fewer virtual servers to provide the same services and allows daytime 
maintenance. 
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2.1 


2.2 


2.3 


Planning for the Virtualized Environment 


Use the information in this section as you plan a Novell Cluster Services cluster that uses virtual 


machines as nodes. 


¢ Section 2.1, "Things to Explore,” on page 17 

¢ Section 2.2, “Infrastructure Dependencies,” on page 17 

* Section 2.3, "LAN and SAN Connectivity,” on page 17 

* Section 2.4, “Time Recommendations," on page 18 

* Section 2.5, “Recommendations,” on page 19 

* Section 2.6, "Upgrading VMware ESXi on the Host Server,” on page 19 

* Section 2.7, "Installing or Updating VMware Tools on Cluster Nodes," on page 20 
¢ Section 2.8, "Known Issues,” on page 20 


Things to Explore 


* VLANS (virtual LANs) allow you to balance performance and fault tolerance. 
* Automated installation 
* Saves time 
* Avoids errors 
* License key 
* Device names 
* Host keys 


Infrastructure Dependencies 


* Name resolution is controlled outside of the virtualization. 
* Time synchronization is controlled outside of the virtualization. 
* Ensure that the host's time is synchronized. 


* Follow the virtualization vendor's time synchronization recommendation documents. 


LAN and SAN Connectivity 


* Use physical device mappings of the NIC and HBA. 
+ And/or use paravirtualization where possible. 
* Install VMware tools. 


* Switch to paravirtualization drivers. 
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2.4 


2.4.1 


2.4.2 


2.4.3 


2.4.4 


2.4.5 


Time Recommendations 


¢ Section 2.4.1, “Time Source,” on page 18 

¢ Section 2.4.2, “UTC Time Format,” on page 18 

* Section 2.4.3, “Tolerance in a Virtualized Environment,” on page 18 
* Section 2.4.4, "Tolerance in a Stretch Cluster,” on page 18 

¢ Section 2.4.5, “Quorum and Quorum Timeout,” on page 18 


¢ Section 2.4.6, “Keep Latency in Mind,” on page 19 


Time Source 


The NCS cluster master node uses its server time to record most log entries. To synchronize time for 
all nodes, you should always use a time source that is external to the cluster. In a virtualized 
environment, the time source must also be external to host servers because the hypervisor provides 
virtual clocks to the guest servers. Using a time source that is external to cluster nodes and host 
servers allows time to be recorded consistently for cluster events. 


UTC Time Format 


UTC (Coordinated Universal Time) is the standard international time. All time zones are defined as an 
offset of UTC. UTC does not get adjusted for daylight savings. Use UTC time format instead of local 
time format on all cluster nodes. In a virtualized environment, ensure that you use UTC time format 
on the host servers and guests servers. 


Tolerance in a Virtualized Environment 


The cluster tolerance setting specifies the amount of time the master node gives all other nodes in the 
cluster to signal that they are alive. In a virtualized environment, a guest server communicates 
through the hypervisor to the network adapter, which can introduce some latency in communications. 
If the delay causes problems for the cluster node, you can increase the tolerance from 8 seconds (the 
default) to 12 seconds. The revised tolerance setting applies to all nodes. If the problem continues, try 
increasing the tolerance another 4 seconds to 16 seconds. 


Tolerance in a Stretch Cluster 


The cluster tolerance setting specifies the amount of time the master node gives all other nodes in the 
cluster to signal that they are alive. In a stretch cluster where nodes are not co-located, the distance 
between nodes can introduce some latency in communications. If the delay causes nodes to not 
meet the tolerance settings for the keep-alive signals, you can increase the tolerance for the cluster. 
Try increasing the tolerance from 8 seconds to 12 seconds. If more time is needed, increase the 
tolerance another 4 seconds to 16 seconds. Try incremental increases until the time allowed is 
sufficient under normal network conditions. The revised tolerance setting applies to all nodes. 


Quorum and Quorum Timeout 


Quorum and Quorum Timeout 
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2.4.6 


2.5 


2.6 


Keep Latency in Mind 


Keep latency in mind. 


Recommendations 


* Use NCS for scalability. 

* Use NCS for service monitoring and service availability. 
* Use NCS for maintenance. 

* Use multipathing on the host. 

* Use LAN fault tolerance on the host. 


* Do not use vMotion for cluster nodes. 


IMPORTANT: If you use vMotion on guest machine that is used as a cluster node, you must 
issue a cluster leave command in the guest environment before the vMotion migration is 
performed in the host environment. 


* Increase tolerance by setting up hardware monitoring to be done by the host. 


* Back up the guest machine with the vm API. Include memory in the backup and keep in mind the 
restore process. For consistency, use the single file restore. 


* No back up to SAN within virtualized environments. 
* Do not create templates or clones of servers that have eDirectory installed. 


* Do not use snapshots of the host. 


IMPORTANT: If you use a host snapshot, ensure that you fix eDirectory when you go back to a 
specific snapshot. 
* Do not mix physical and virtual cluster nodes. 


Physical and virtual cluster nodes will have different response times and performance 
characteristics. It is preferred to have consistency of hardware or guest environments for nodes 
in the same cluster. 


* Do not run multiple NCS nodes on the same virtualization host. 


The server hardware represents a single point of failure for all virtual machines running on the 
same virtualization host. 


* All nodes require the same view to LAN and storage. 
* No I/O caching to the SBD. 
* Add multiple CPUs and enough memory. 


Upgrading VMware ESXi on the Host Server 


Before you upgrade VMware ESXi on a host server, you must first shut down all guest servers. A 
clustered guest server on the host is not available to its NCS cluster during the host upgrade process. 
The node will rejoin the cluster when it is rebooted after the host upgrade. 
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2.1 


2.8 


Installing or Updating VMware Tools on Cluster 
Nodes 


Installing or updating VMware tools on a clustered guest server might cause brief interruptions to vital 
system services and indicators that are closely watched by NCS. As a result, the guest server might 
be fenced during the process. To avoid such problems, you must stop NCS on a node before you 
install or update VMware tools on it. You can start NCS on the node after you have successfully 
installed or updated VMware tools. 


Alternatively, you can put the whole cluster in maintenance mode before you install or update 
VMware tools on the clustered guest servers, and disable maintenance mode afterwards. However, if 
something goes wrong with VMware tools update or installation, the cluster might be left in a 
unknown state, and you might need to reboot some nodes in order to bring the cluster out of 
maintenance mode. 


Known Issues 


* NIC status is insulated by virtual switches. For faster responses, use dedicated NICs for the 
virtual machine. 


In VMware, a dedicated NIC is one that is presented exclusively to one guest operating system 
through pass-through PCI/PCle. The number of dedicated NICs cannot exceed the number of 
NICs the physical box has. 


* VMware presents NICs as pass-through PCI/PCIe. The virtual machine does not work directly 
with the NIC. 


Different NIC orders can occur with pass-through PCI/PCIe. 


* 


* 


Delay in reporting SAN errors. 


* 


Deletion of RDM. You must delete virtual machines manually. 
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3.1 


Implementing Novell Cluster Services in 


an ESXi Virtualized Environment 


This section describes how to configure virtual machines on an ESXi hypervisor to maximize their 
performance when used with Novell Cluster Services for Novell Open Enterprise Server (OES) 11 
SP3. 


* 


* 


* 


* 


* 


Section 3.1, "Supported Configurations," on page 21 

Section 3.2, "Required Software," on page 25 

Section 3.3, "Installation Overview," on page 25 

Section 3.4, "Setting Up Virtual Switches on the ESXi Host," on page 26 
Section 3.5, "Setting Up Dedicated NICs for a Virtual Machine,” on page 26 
Section 3.6, "Setting Up a Time Server for a Virtual Machine," on page 27 
Section 3.7, "Preparing Shared Storage," on page 28 

Section 3.8, "Creating a Virtual Machine," on page 30 

Section 3.9, "Guest Operating System Installation Sources," on page 55 


Supported Configurations 


A typical Novell Cluster Services cluster consists of a number of machines that coordinate through 


computer networks to make resources highly available. Machines can be physical, virtual nodes on a 
single VMware host, virtual nodes on different VMware hosts, or any combination of those. 


This section illustrates the supported configurations. These examples are not intended to be limiting 
or an exhaustive representation of all possible combinations. 


* 


* 


* 


* 


Section 3.1.1, "Physical Machines," on page 22 

Section 3.1.2, "Virtual Machines on a Single VMware Host," on page 22 
Section 3.1.3, "Virtual Machines from Different VMware Hosts," on page 23 
Section 3.1.4, “Mixtures or Subsets of All of the Above,” on page 23 
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3.11 Physical Machines 


In Figure 3-1, the NCS cluster nodes are all physical machines. The websites are cluster resources 
on shared disks that can fail over between the physical nodes. All nodes are running the same 
version of the OES operating system. 


Figure 3-1 NCS Cluster with Only Physical Nodes 
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3.1.2 Virtual Machines on a Single VMware Host 


In Figure 3-2, the NCS cluster nodes are all virtual machines. Each node is hosted on the same 
VMware host. All nodes are running the same version of the OES operating system. 


Figure 3-2 NCS Cluster with Virtual Nodes from a Single VMware Host 
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3.13 Virtual Machines from Different VMware Hosts 


In Figure 3-3, the NCS cluster nodes are all virtual machines. Each node is hosted on a different 
VMware host. All nodes are running the same version of the OES operating system. 


Figure 3-3 NCS Cluster with Virtual Nodes from Different VMware Hosts 
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3.1.4 Mixtures or Subsets of All of the Above 


In Figure 3-4, the NCS cluster combines physical nodes with a virtual node from a VMware host. All 
nodes are running the same version of the OES operating system. 


Figure 3-4 NCS Cluster with Physical Nodes and a Virtual Node 
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In Figure 3-5, nodes from multiple clusters reside on the same VMware host. Multiple two-node 
clusters are formed by combining a virtual node from one VMware host with a virtual node from a 
different VMware host. All nodes in the same cluster are running the same version of the OES 
operating system. 


Figure 3-5 NCS Two-Node Clusters with Virtual Nodes from Different VMware Hosts 
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3.2 


3.3 


In Figure 3-6, nodes from multiple clusters reside on the same VMware host. Multiple two-node 
clusters are formed by combining a physical node with a virtual node from the VMware host. All 
nodes are running the same version of the OES operating system. 


Figure 3-6 NCS Two-Node Clusters with Each Physical Node Paired a Different Virtual Node from a Single 
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Required Software 


The implementation described in this guide requires the following software to create the virtual 
machines: 


* VMware ESXi, including SAN and LAN driver software 
* VMware vSphere client 


Installation Overview 


To set up your virtualized environment: 


1 Install the ESXi hosts, including the SAN and LAN driver software. 
2 Configure the networks. 
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3 Prepare the SAN. 


3a Set up two disks that you will use for the split-brain detector (SBD) for the cluster. 


This allows you to mirror the SBD for cluster fault tolerance. 


3b Setup the shared storage and allocate the disks to the ESXi host. 


4 Create 64-bit virtual machines. 


5 Install OES on the virtual machines. 


6 Configure Novell Cluster Services to create a cluster on the first node, then on the other nodes to 


add them to the cluster. 


3.4 Setting Up Virtual Switches on the ESXi Host 


Figure 3-7. Virtual Switch Example 
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35 Setting Up Dedicated NICs for a Virtual Machine 


Mark the NICS as VMDirectPath I/O pass-through devices on the host: 


1 Select an ESX\ESXi host from the Inventory of VMware vSphere Client. 


2 In the Configuration tab, click Advanced Settings. The passthrough Configuration page lists all 
available pass-through devices, as shown in the picture. 
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Figure 3-8 Dedicated NIC Example 
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If the NICs are not listed, you need to click Edit, select the NICs and click OK. 


When the NICs are selected, they are marked with an orange icon. Reboot for the change to 
take effect. After rebooting, the NICs are marked with a green icon and are enabled to be used 
as dedicated NICs. 


Assign the NICs to VMs: 


1 From the Inventory in vSphere Client, right-click the virtual machine and click Edit Settings. 
2 Click the Hardware tab. 

3 Click Add. 

4 Choose the NICs. 

5 Click Next. 


3.6 Setting Up a Time Server for a Virtual Machine 


To make a VM refer to single Time server: 


1 In the vSphere client, click the Configuration tab. 

2 In the left navigation under Software, click Time Configuration. 

3 Select Properties » Options. 

4 Click Add, specify the IP address of the time server, then click OK. 
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3.7 Preparing Shared Storage 


1 Ensure that your shared disks are visible to the VMware hosts. You can use the vSphere Client 
or the command line interface. 


* vSphere Client: In the vSphere client, go to Host » Configuration » Storage » Devices. 
For example, Figure 3-9 shows an example list of devices as viewed from the vSphere 
client: 


Figure 3-9 List of Shared Devices on the ESXi Host as Viewed in the VSphere Client 
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* Command Line Interface: As the root user, launch a terminal console, then enter 


ls -1Sh /vmfs/devices/disks/* | grep -v '^lV|:[0-9]M«$' 
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For example, Figure 3-10 shows the example output from this command: 


Figure 3-10 List of Shared Devices on the ESXi Host as Viewed in the ESXi Shell 


2 For each disk that you want to use with the virtual machines, create a virtual compatible Raw 
Disk Mapping (RDM). Store the RDMs outside of the virtual machines. 


It is faster to use the command line when mapping the devices. If you use the vSphere Client, 
you must create the RDMs when you add the first virtual machine on each host. Using the client 
is time-consuming and error-prone. 


If your virtual machines reside on multiple hosts, you can generate a script to perform the 
preceding steps on one host, then run it on all the hosts. 


2a Create the location where you will store the RDMs. At a command prompt, enter 
mkdir -p /vmfs/volumes/«datastore name»/sharedDisks 


Replace datastore name With the name of the datastore you are creating. 


For example, enter 
mkdir -p /vmfs/volumes/datastorel/sharedDisks 


2b Map each disk to the RDM location. At a command prompt, enter the following command for 
each disk: 


vmkfstools -r «disk name» «RDM location» 
For example, enter 


vmkfstools -r /vmfs/devices/disks/naa.60060160b6561601dedf0855331de211 /vmfs/volumes/ 
datastorel/sharedDisks/VM-SAN-512G-NSS.vmdk 


Implementing Novell Cluster Services in an ESXi Virtualized Environment 29 


30 


Figure 3-11 Commands to Map Disks to the RDM Location 
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#1039 root @CGServer:/usr/src/cont/vmuare # lj 


2c Repeat this process for each virtual node in the cluster. 
3 Eager-zero the SBD devices. At a command prompt, enter 


vmkfstools -k sbd rdm 


Eager-zero writes zeroes to all allocated blocks on the device. This allocates all required space 
for the virtual disk at this time. It might take several minutes to zero-out the devices. 


38 Creating a Virtual Machine 


As you create the virtual machine by using the vSphere Client, many settings will use the default 
value. Ensure that you use the following recommended settings: 


* For the iSCSI controller, use LSI Logic SAS (the default controller type for SUSE Linux 
Enterprise Server 11 64-bit). 


NOTE: VMware Paravirtual SCSI controllers cannot be used in clustering environments because 
they use caches to achieve better throughput. 


* Add virtual NICs if needed. 


You can specify whatever NIC type you need based on your hardware, network needs, and other 
considerations for your environment. 


For example, E1000 and VMXNET3 NIC types have drivers in the Linux kernel and do not 
require VMware Tools to be installed in order to run. This allows these NICs to be available 
during the operating system installation. 


* Edit the VM settings before you complete the virtual machine creation. 
* Add the shared storage. 
* Change the MAC address setting to Manual. 
* Remove unwanted hardware, such as the floppy disk. 
* Add dedicated NICs. 
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Ensure that you meet the prerequisites for creating virtual machines, as described in "Start the Virtual 
Machine Creation Process in the vSphere Client" (http://pubs.vmware.com/vsphere-50/ 
index.jsp?topic-962Fcom.vmware.vsphere.vm admin.doc 50962FGUID-4COF4D73-82F2-4B81- 
8AAT-1DD752A8A5AC.html) in the VMware vSphere 5 Documentation Center. 

1 Launch the vSphere Client. 


2 In the vSphere Client, display the inventory objects in the vSphere Client by using the VM and 
Templates view. 


3 Right-click an object and select New » Virtual Machine. 
4 On the Configuration page, select Custom, then click Next. 


Figure 3-12 Create New VM: Configuration - Custom 
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5 On the Name and Location page, select a name and location for the virtual machine, then click 
Next. 


The name you enter is used as the virtual machine's base name in the inventory and for the 
name of the virtual machine's files. The name must be unique on the host server. Names are 
case insensitive and can be up to 80 characters long. 


Figure 3-13 Create New VM: Name and Location 
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6 On the Storage page, select a destination datastore for the virtual machine files, then click Next. 
For this example, select datastore1. 


Figure 3-14 Create New VM: Storage 
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7 On the Virtual Machine Version page, specify the compatibility version desired for the virtual 
machine image, then click Next. 


If the host supports more than one VMware virtual machine version, you can select a version for 


the virtual machine. 


Figure 3-15 Create New VM: VM Version 
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m Virtual Machine Version 


This host or cluster supports more than one VMware virtual machine version. Specify the virtual 
machine version to use. 


C Virtual Machine Version: 4 


This version will run on VMware ESX 3.0 and later, and VMware Server 1.0 and later. This 
version is recommended when sharing storage or virtual machines with ESX up to 3.5. 


C Virtual Machine Version: 7 


This version will run on VMware ESX/ESXI 4.0 and later. This version is recommended when 
sharing storage or virtual machines with ESX/ESXi up to 4.1. 


(* Virtual Machine Version: 8 


This version will run on VMware ESXi 5.0 and later. Choose this version iF you need the latest 
virtual machine features and do not need to migrate to ESX/ESXI 4. 
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8 Specify the guest operating system and version to use with this virtual machine, then click Next. 


For the OES Server, select Linux as the Guest Operating System, then select SUSE Linux 
Enterprise Server 11 64-bit from the Version drop-down list. 


Figure 3-16 Create New VM: Guest Operating System 
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Ready to Complete Identifying the guest operating system here allows the wizard to provide the appropriate defaults for 
the operating system installation. 
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9 On the CPUs page, select the number of virtual sockets for the virtual machine, select the 
number of cores per virtual socket, then click Next. 


The total number of cores is the number of cores per socket multiplied by the number of virtual 
Sockets. The resulting total number of cores should be a number equal to or less than the 
number of logical CPUs on the host. 


Figure 3-17 Create New VM: CPUs 
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10 On the Memory page, specify the amount of memory that the guest operating system can use, 
then click Next. 


The minimum memory size is based on the type of firmware used by the virtual machine: 
* BIOS firmware: 4 MB 
* EFI firmware: 96 MB 
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The maximum memory size for a virtual machine depends on the host's physical memory and 
the virtual machine's hardware version. The size must be a multiple of 4 MB. 


Figure 3-18 Create New VM: Memory 
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11 On the Network page, specify 2 as the number of virtual NICs to connect, specify the following 
values for each NIC, then click Next. 
NIC Network NIC Connect at Power On 
NIC1 VM Public E1000 Selected 


(or as appropriate) 


NIC2 VM Private E1000 Selected 


(or as appropriate) 


For the public IP address, you can use any static public IPv4 address that is available to your 
company. 


In VMware, a private NIC is a virtual NIC connected to virtual switch that is on a private network. 
You can have as many private NICs as the VMware host can handle. You can use any available 
private IP address to bond to the private NIC. For IPv4, the private IP addresses are 10.0.0.0 to 
10.255.255.255, 172.16.0.0 to 172.31.255.255, and 192.168.0.0 to 192.168.255.255. 
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IMPORTANT: NCS does not require private NICs or private networks, whether you are using 
virtual or physical environments. 


There are perceived advantages and disadvantages for using private NICs. With private NICs, 
your NCS inter-node traffic can be kept internal and separate from data access traffic, which 
keeps it more secure and provides separate bandwidth. However, the separation could create a 
scenario where NCS traffic moves smoothly while data access traffic is down. Some 
administrators prefer to deliberately combine NCS and client traffic together to avoid this kind of 
problem. 


Figure 3-19 Create New VM: Network 
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Ew If supported by this virtual machine version, more than 4 NICs can be added after the 
virtual machine is created, via its Edit Settings dialog. 


Adapter choice can affect both networking performance and migration compatiblity. Consuk 
the VMware KnowledgeBase for more information on choosing among the network adapters 
supported for various guest operating systems and hosts. 
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12 On the SCSI Controller page, select the SCSI controller type as LSI Logic SAS (the default for 
SLES 11 64-bit), then click Next. 


The wizard preselects the correct default controller based on the guest operation system you 
selected on the Guest Operating System page. 


Figure 3-20 Create New VM: SCSI Controller 
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13 On the Select a Disk page, select Create a new virtual disk as the virtual disk type to use for the 
system device, then click Next. 


Figure 3-21 Create New VM: Disk 
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Select the type of disk to use. 
Disk 
(V Create a new virtual disk 


C Use an existing virtual disk 
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Give your virtual machine direct access to SAN. This option allows you to 
use existing SAN commands to manage the storage and continue to 
access it using à 


Do not create disk 


14 On the Create a Disk page, specify the following information for the virtual disk, then click Next. 
* Disk Size: Specify the amount of space to use for the system device. For example, 40 GB. 
* Disk Provisioning: Select Thick Provision Lazy Zero. 


The specified disk space for the virtual disk is allocated at create time. Any data remaining 
on the physical device is zeroed out on demand at a later time on first write from the virtual 
machine. 


* Disk Location: Select Store with the virtual machine. 
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Figure 3-22 Create New VM: Create a Disk 
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© Store with the virtual machine 


C Specify a datastore or datastore cluster: 
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15 On the Advanced Options page, specify the advanced options to use for the virtual disk, then 
click Next. 


* Virtual Device Node: Select SCSI (0:0) from the drop-down box. 
* Mode: Deselect the Independent check box. 


Figure 3-23 Step 15 (Advanced Options) 
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16 On the Ready to Complete page, review the configuration settings, select Edit the virtual 
machine settings before completion, then click Continue. 


Figure 3-24 Create New VM: Ready to Complete 
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IV Edit the virtual machine settings before completion 


Á Creation of the virtual machine (VM) does not include automatic installation of the guest operating 
system. Install a guest OS on the VM after creating the VM. 


17 On the Hardware tab of the Virtual Machine Properties page, review the hardware properties and 
edit the VM settings before you complete the virtual machine creation. 


The procedures in this step describe how to modify the following settings: 
* Add the shared storage. 
* Add a CD/ROM device if you are installing from the host CDROM device or from an ISO file. 
* Remove unwanted hardware, such as the floppy disk. 
* Setthe BIOS screen to appear on the next reboot. 
* Change the MAC address setting to Manual and use static MAC addresses. 
Add dedicated NICs. 


* 
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Figure 3-25. Create New VM: VM Properties 


m Memory Configuration 


1011 GB Memory Size: | 4] [ez] 


512 @ Maximum recommended for this 
256 GB "4 guest OS: 1011 GB. 


CPUs (adding) Maximum recommended for best 
Video card (adding) 128 GB * performance: 12276 MB. 
VMCI device (adding) Restricted 64 GB Default Ecce for this 
New CD/DVD (adding) Client Device 32 GB d itas 

Minimum recommended for this 
New Floppy (adding) Client Device 16 GB guest OS: 512 MB. 
New SCSI Controller (add... ^ LSI Logic SAS 
New NIC (adding) ¥M Public 
New NIC (adding) ¥M Private 4GB 
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17a Add the shared storage that you created and mapped in Section 3.7, "Preparing Shared 
Storage," on page 28. 


Repeat the following procedure for each shared hard disk that you want to add to this virtual 
machine: 


17a1 On the Hardware tab, click Add, select Hard Disk as the Device Type, then click Next. 
You can also select New Hard Disk, then click Add. 


Figure 3-26 Add Hardware: Add» Device Type » Hard Disk 
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Device Type 
What sort of device do you wish to add to your virtual machine? 


Device Type 
Select a Disk 
Create a Disk 
Advanced Options 
Ready to Complete 


Choose the type of device you wish to add. 


USB Device (unavailable) 
PCI Device (unavailable) 


m Information — — ————————————————34 


This device can be added to this Virtual Machine. 
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17a2 On the Select a Disk page, select Use an existing virtual disk, then click Next. 


Figure 3-27 Add Hardware: Select a Disk 


Device Type A virtual disk is composed of one or more files on the host file system. Together these 
Select a Disk files appear as a single hard disk to the guest operating system. 

Select Existing Disk 

Advanced Options Select the type of disk to use. 


Ready to Complete Disk 


C Create a new virtual disk 


@ Use an existing virtual disk 
Reuse a previously configured virtual disk. 


C Raw Device Mappings 
Give your virtual machine direct access to SAN. This option allows you to 
use existing SAN commands to manage the storage and continue to 
access it using a datastore. 
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17a3 On the Select Existing Disk page, click Browse. In the pop-up browser, locate the 
datastore, select the disk you want to add from the datastore, then click OK. On the 
Select Existing Disk page, click Next. 


As a best practice for consistency, add the SBD disks first, then add the data disks. For 
the first hard disk you add, select one of the disks you prepared for use as an SBD. For 
the second hard disk you add, select the second disk that you prepared for the SBD. 
Thereafter, you can add data disks in your preferred order. 


Figure 3-28 Add Hardware: Select Existing Disk 


Select Existing Disk 
Which existing disk do you want to use as this virtual disk? 


oa Disk File Path 


Select a Disk 

Select Existing Disk | 
Advanced Options 

Ready to Complete 
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Figure 3-29 Add Hardware: Browse Datastores 


11/9/2012 10:21:38 AM 
1/7/2013 12:03:28 PM 
1/5/2013 10:15:15 PM 
1/5/2013 10:14:53 PM 
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11/9/2012 10:21:59 AM 
1/5/2013 9:43:59 PM 
1/7/2013 8:53:59 AM 
11/9/2012 10:22:08 AM 
11/9/2012 10:22:10 AM 
11/9/2012 10:22:13 AM 
11/9/2012 10:22:16 AM 
11/9/2012 10:22:19 AM 
11/9/2012 10:22:22 AM 
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1/5/2013 10:41:39 PM 
1/5/2013 10:31:24 PM 
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Compatible Virtual Disks (*.vmdk, *.dsk, *.raw) 


17a4 On the Advanced Options page, specify the following values, then click Next. 


* Virtual Device Node: Select SCSI (1:0) (or the default node value offered as 
appropriate if you have added multiple disks) from the drop-down box. 


* Mode: Select the Independent check box, then select Persistent. 


Disks in persistent mode behave like conventional disks on your physical 
computer. All data written to a disk in persistent mode are written permanently to 
the disk. 
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Figure 3-30 Add Hardware: Advanced Options for the Existing Disk 


Advanced Options 
These advanced options do not usually need to be changed. 


Specify the advanced options for this virtual disk. These options do not normally need 
to be changed. 


Select Existing Dis 
Advanced (pss 


Ready to Complete [scs (1:0) v | 


IV Independent 
Independent disks are not affected by snapshots. 
(* Persistent 
Changes are immediately and permanently written to the disk. 


C Nonpersistent 
Changes to this disk are discarded when you power off or revert to the 
snapshot. 
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17a5 On the Ready to Complete page, review the selected options, then click Finish. 


Figure 3-31 Add Hardware: Ready to Complete Add Existing Hard Disk 


Ready to Complete 
Review the selected options and click Finish to add the hardware. 


Hard Disk 
Use existing disk 
: SCSI (1:0) 
Ready to Complete i : [datastore1] sharedDisks/VM-SAN-001G-SBD-A.vmdk 
Independent-persistent 


17a6 Repeat the previous steps for each shared hard disk that you want to add. 
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17b On the Hardware tab of the Virtual Machine Properties page, add a CD/DVD device, then 
set its properties. 


* Device Status: Select the Connect at power on check box. 
* Device Type: Use one of the following resources: 


If you are installing from a DVD, select Host device, then select the path of the device 
from the drop-down list. In the Figure 3-32 example, the path is /vmfs/devices/ 
cdrom/mpx.vmhba0:CO0:T0:L0. 


If you are installing from a DVD ISO file, select Datastore ISO file, then browse to 
locate and select the file. 


* Virtual Device Node: Select the device node from the drop-down list, such as IDE 
(1:0). 


Figure 3-32 Create New VM: CD/DVD Disk Properties 


| Hardware | Options | Resources | 
Device Status 


Bros L9. | me | |r 
; [V Connect at power on 

NN Memory (adding) 4096 MB 

[] CPUs (adding) 4 

(LJ video card (adding) Video card 

© YMCI device (adding) Restricted 

(Qj New CD/DVD (adding) /vmfs/devices/cdr.. | 

& New Floppy (adding) Client Device © is bone 

© New SCSI Controller (add... LSI Logic SAS 

WB New NIC (adding) IM Public [Ivmfsjdevicesicdromimpx.vmhba0:C0:T0:LO X | 

WE New NIC (adding) ¥M Private C Datastore ISO File 

© New Hard Disk (adding) Virtual Disk 

© New SCSI Controller (add... LSI Logic SAS 

@ New Hard Disk (adding) Mapped Raw LUN 


Device Type 
C Client Device 


Virtual Device Node 
@ [IDE (1:0) 


17c On the Hardware tab of the Virtual Machine Properties page, remove unwanted hardware, 
such as the floppy disk. Select the object, then click Remove. 
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17d On the Options tab of the Virtual Machine Properties page, review the settings. If you need 
to set the BIOS for the virtual machine, select the Force BIOS Setup check box. 


The next time the virtual machine boots, it is forced into the BIOS setup screen. 


Figure 3-33 Create New VM: Options » Boot Options 


MyVM001 - Virtua 
Firmware 
Specify the boot firmware: 
(* gos 
C gri 


Changing firmware may cause installed guest operating system 
to become unbootable. 


Expose Nx flagto ... 


abled Power On Boot Delay 


Whenever the virtual machine is powered on or reset, 


Boot to BIOS delay the boot for the following number of miliseconds: p 3 


-Force BIOS Setup 


I") The next time the virtual machine boots, force entry into the 
BIOS setup screen. 


Failed Boot Recovery 
[^ when the virtual machine fails to find boot device, 
automaticaly retry boot after |10 =] seconds 
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17e On the Hardware tab of the Virtual Machine Properties page, select the VM Private NIC, 
change the MAC Address setting to Manual, then type the static MAC address. 


Figure 3-34 Create New VM: Hardware » NIC » Manual MAC Adaress 


oltuu 


CPUs (adding) 

Video card (adding) 

VMCI device (adding) 

New CD/DVD (adding) 

New Floppy (adding) 

New SCSI Controller (add... 


Restricted 
Client Device 
Client Device 

LSI Logic Parallel 


Adapter Type 
Current adapter: 


MAC Address 
[00:50:56:00:00:01 


C Automatic (* Manual 


New NIC (adding) 


VM Private 


8 


New Hard Disk (adding) 


Virtual Disk 


DirectPath 1/0 


Status: 


r- Network Connection 


Network label: 


[vm Private y | 
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17f On the Hardware tab of the Virtual Machine Properties page, add dedicated NICs. 
17f1 On the Hardware tab, click Add. 
17f2 On the Add Hardware page, click PCI Device. 


Figure 3-35 Add Hardware: Choose PCI Device 


Which of the present PCIJPCIe devices would you like to add? 


Device Type | Connection — 
Select PCI/PCIe Device , 
Ready to Complete Specify the physical PCI/PCIe Device to connect to: 


[04:00.0 | Broadcom Corporation Broadcom Netxtreme II BCMS709 10008ase -| 


Note: The presence of a PCI/PCIe device passthrough wil 

A prevent the use of many commands on the virtual machine. 
It will not be able to be suspended, to have snapshots taken 
or restored, or to participate in vMotion. 


rg; Adding a PCI Passthrough device to this VM will automatically 
CX set its minimum memory reservation equal to its memory size. 


17f3 On the Chose PCI Device page, specify the physical NIC to use for the NIC. 
17f4 Click Next. 
17g Click Finish. 


18 In the vSphere client, select Host » Virtual Machines, then verify that the newly created virtual 
machine appears in the VM list of the vSphere client. 


Figure 3-36 VM List in the VSphere Client 
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(Optional) View a list of the files created for the virtual machine. In a terminal console, go to the 
location that you specified for the virtual machine, then enter 


ls -1 

For example: 

#1 root@CG-ESXi-02:/vmfs/volumes/509d3948-7ed53294-d0b3-984 
-rw------- 1 root root 0 Nov 27 21:54 
-rw------- 1 root root 42949672960 Nov 27 22:37 
-rw-r--r-- 1 root root 8684 Nov 27 22:34 
-rw------- 1 root root 466 Nov 27 21:55 
-rw-r--r-- 1 root root 0 Nov 9 19:2 

-rw-r--r-- 1 root root 6862 Nov 27 21:54 
spyweens-2-- 1 root root 0 Nov 9 19:2 

-rw-r--r-- 1 root root 256 Nov 9 19:2 

-rw-r--r-- lwroot root 6862 Nov 27 21:54 
-rw-r--r-- 1 root root 2662757 Nov 27 22:04 
-rw------- 1 root root 8566144 Nov 9 19:2 


be1751d14/CG-02 # 1s -1 


CG-02-8ddbb415.vswp 
CG-02-flat.vmdk 
CG-02.nvram 
CG-02.vmdk 
CG-02.vmsd 
CG-02.vmx 
CG-02.vmx.lck 
CG-02.vmxf 
CG-02.vmx- 
vmware.log 
vmx-CG-02-2379985941-1.vswp 


3.9 Guest Operating System Installation Sources 


You can install OES as a guest operating system on the virtual machine by using one of the following 
sources: 


* 


CD/DVD device 


When you use the host's physical CD/DVD device, only one VM can access the device at a time. 


ISO image as CD/DVD device 
Network repository / AutoYaST 


USB 
PXE Boot 
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Documentation Updates 


This section contains information about documentation content changes made to the Novell Cluster 
Services Implementation Guide for VMware since the initial 2.0 release for Novell Open Enterprise 
Server 11. 


This document was updated on the following dates: 


* Section A.1, "March 2015 (OES 11 SP2),” on page 57 
* Section A.2, "January 2014 (OES 11 SP2),” on page 57 


A.1 March 2015 (OES 11 SP2) 


Updated Section 3.5, "Setting Up Dedicated NICs for a Virtual Machine," on page 26 with procedural 
instructions. 


A.2 January 2014 (OES 11 SP2) 


Modified eDirectory links to point to the NetIQ eDirectory 8.8 website (http://www.netiq.com/ 
documentation/edir88). 


Updates were made to the following sections. The changes are explained below. 


¢ Section A.2.1, "Implementing Novell Cluster Services in an ESXi Virtualized Environment,” on 
page 57 


* Section A.2.2, "Planning for the Virtualized Environment," on page 58 


A.2.1 Implementing Novell Cluster Services in an ESXi Virtualized 


Environment 
Location Change 
Section 3.6, "Setting Up a Time Server for a Virtual This section is new. 


Machine," on page 27 
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A.2.2 


Location 


Section 3.8, "Creating a Virtual Machine," on page 30 


Change 


* iSCSI controller 


NOTE: VMware Paravirtual SCSI controllers 
cannot be used in clustering environments 
because they use caches to achieve better 
throughput. 


* NIC types 


You can specify whatever NIC type you need 
based on your hardware, network needs, and 
other considerations for your environment. 


For example, E1000 and VMXNETS3 NIC types 
have drivers in the Linux kernel and do not 
require VMware Tools to be installed in order to 
run. This allows these NICs to be available during 
the operating system installation. 


Planning for the Virtualized Environment 


Location 


Section 2.6, "Upgrading VMware ESXi on the Host 
Server," on page 19 


Section 2.7, "Installing or Updating VMware Tools on 


Cluster Nodes," on page 20 


Change 


This section is new. 


Stop Novell Cluster Services on a clustered guest 
server before you install or update VMware tools. 
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